GBM-based Bregman Proximal Algorithms for Constrained Learning

As the complexity of learning tasks surges, modern machine learning encounters a new constrained learning paradigm characterized by more intricate and data-driven function constraints. Prominent applications include Neyman-Pearson classification (NPC) and fairness classification, which entail specific risk constraints that render standard projection-based training algorithms unsuitable. Gradient boosting machines (GBMs) are among the most popular algorithms for supervised learning; however, they are generally limited to unconstrained settings. In this paper, we adapt the GBM for constrained learning tasks within the framework of Bregman proximal algorithms. We introduce a new Bregman primal-dual method with a global optimality guarantee when the learning objective and constraint functions are convex. In cases of nonconvex functions, we demonstrate how our algorithm remains effective under a Bregman proximal point framework. Distinct from existing constrained learning algorithms, ours possess a unique advantage in their ability to seamlessly integrate with publicly available GBM implementations such as XGBoost (Chen and Guestrin, 2016) and LightGBM (Ke et al., 2017), exclusively relying on their public interfaces. We provide substantial experimental evidence to showcase the effectiveness of the Bregman algorithm framework. While our primary focus is on NPC and fairness ML, our framework holds significant potential for a broader range of constrained learning applications. The source code is currently freely available at https://github.com/zhenweilin/ConstrainedGBM}{https://github.com/zhenweilin/ConstrainedGBM

Efficient First-order Methods for Convex Optimization with Strongly Convex Function Constraints

Convex function constrained optimization has received growing research interests lately. For a special convex problem which has strongly convex function constraints, we develop a new accelerated primal-dual first-order method that obtains an \Ocal(1/\sqrt{\vep}) complexity bound, improving the \Ocal(1/{\vep}) result for the state-of-the-art first-order methods. The key ingredient to our development is some novel techniques to progressively estimate the strong convexity of the Lagrangian function, which enables adaptive step-size selection and faster convergence performance. In addition, we show that the complexity is further improvable in terms of the dependence on some problem parameter, via a restart scheme that calls the accelerated method repeatedly. As an application, we consider sparsity-inducing constrained optimization which has a separable convex objective and a strongly convex loss constraint. In addition to achieving fast convergence, we show that the restarted method can effectively identify the sparsity pattern (active-set) of the optimal solution in finite steps. To the best of our knowledge, this is the first active-set identification result for sparsity-inducing constrained optimization.Comment: 27 pages, 3 figure

Decentralized Gradient-Free Methods for Stochastic Non-Smooth Non-Convex Optimization

We consider decentralized gradient-free optimization of minimizing Lipschitz continuous functions that satisfy neither smoothness nor convexity assumption. We propose two novel gradient-free algorithms, the Decentralized Gradient-Free Method (DGFM) and its variant, the Decentralized Gradient-Free Method$^+$ (DGFM$^{+}$). Based on the techniques of randomized smoothing and gradient tracking, DGFM requires the computation of the zeroth-order oracle of a single sample in each iteration, making it less demanding in terms of computational resources for individual computing nodes. Theoretically, DGFM achieves a complexity of $\mathcal O(d^{3/2}\delta^{-1}\varepsilon ^{-4})$ for obtaining an $(\delta,\varepsilon)$-Goldstein stationary point. DGFM$^{+}$, an advanced version of DGFM, incorporates variance reduction to further improve the convergence behavior. It samples a mini-batch at each iteration and periodically draws a larger batch of data, which improves the complexity to $\mathcal O(d^{3/2}\delta^{-1} \varepsilon^{-3})$. Moreover, experimental results underscore the empirical advantages of our proposed algorithms when applied to real-world datasets

Long-Range Grouping Transformer for Multi-View 3D Reconstruction

Nowadays, transformer networks have demonstrated superior performance in many computer vision tasks. In a multi-view 3D reconstruction algorithm following this paradigm, self-attention processing has to deal with intricate image tokens including massive information when facing heavy amounts of view input. The curse of information content leads to the extreme difficulty of model learning. To alleviate this problem, recent methods compress the token number representing each view or discard the attention operations between the tokens from different views. Obviously, they give a negative impact on performance. Therefore, we propose long-range grouping attention (LGA) based on the divide-and-conquer principle. Tokens from all views are grouped for separate attention operations. The tokens in each group are sampled from all views and can provide macro representation for the resided view. The richness of feature learning is guaranteed by the diversity among different groups. An effective and efficient encoder can be established which connects inter-view features using LGA and extract intra-view features using the standard self-attention layer. Moreover, a novel progressive upsampling decoder is also designed for voxel generation with relatively high resolution. Hinging on the above, we construct a powerful transformer-based network, called LRGT. Experimental results on ShapeNet verify our method achieves SOTA accuracy in multi-view reconstruction. Code will be available at https://github.com/LiyingCV/Long-Range-Grouping-Transformer.Comment: Accepted to ICCV 202

GARNet: Global-Aware Multi-View 3D Reconstruction Network and the Cost-Performance Tradeoff

Deep learning technology has made great progress in multi-view 3D reconstruction tasks. At present, most mainstream solutions establish the mapping between views and shape of an object by assembling the networks of 2D encoder and 3D decoder as the basic structure while they adopt different approaches to obtain aggregation of features from several views. Among them, the methods using attention-based fusion perform better and more stable than the others, however, they still have an obvious shortcoming -- the strong independence of each view during predicting the weights for merging leads to a lack of adaption of the global state. In this paper, we propose a global-aware attention-based fusion approach that builds the correlation between each branch and the global to provide a comprehensive foundation for weights inference. In order to enhance the ability of the network, we introduce a novel loss function to supervise the shape overall and propose a dynamic two-stage training strategy that can effectively adapt to all reconstructors with attention-based fusion. Experiments on ShapeNet verify that our method outperforms existing SOTA methods while the amount of parameters is far less than the same type of algorithm, Pix2Vox++. Furthermore, we propose a view-reduction method based on maximizing diversity and discuss the cost-performance tradeoff of our model to achieve a better performance when facing heavy input amount and limited computational cost

The value of a novel percutaneous lung puncture clamp biopsy technique in the diagnosis of pulmonary nodules

Abstract Background: Computed tomography-guided percutaneous lung biopsy is a crucial method to determine pulmonary anomalies, and is highly accurate in detecting evidence of malignancies, allowing medical practitioners to identify the stage of malignancy and thus help to plan the treatment regimens of patients.Objective: To explore the clinical application of a new computed tomography-guided percutaneous lung puncture clamp biopsy technique in the diagnosis of pulmonary nodules, characterized by ground-glass opacity on chest computed tomography images.Methods: A unique instrument named ‘combined percutaneous lung biopsy forceps’, consisting of a biopsy forceps, a 15-gauge coaxial needle and needle core, was designed. The new tool was used to obtain specimens in nine patients with pulmonary ground-glass opacity. The specimen volumes and the safety of using the instrument were measured. The samples obtained were also assessed to see if they were sufficient for conducting histological tests.Result: Samples were obtained in all nine patients – a success rate of 100%. Consistently, the volume of each specimen was sufficient to make a histological diagnosis. No serious complications, such as pneumothorax – primary spontaneous pneumothorax or secondary spontaneous pneumothorax – occurred during the biopsy.Conclusions: The application of this new tool in obtaining tissue specimens in patients with pulmonary ground-glass opacity under the guidance of chest computed tomography was invaluable in terms of its high accuracy and safety. Moreover, its effect was better compared to using a fine-needle aspiration biopsy or a cutting-needle biopsy. Therefore, this instrument can be used for histological diagnosis. [Ethiop. J. Health Dev. 2021; 35(2):85-90]Key words: Ground-glass opacity; percutaneous lung puncture clamp biopsy; fine-needle aspiration biopsy; cutting-needle biops

Atrial fibrillation and electrophysiology in transgenic mice with cardiac-restricted overexpression of FKBP12

Cardiomyocyte-restricted overexpression of FK506-binding protein 12 transgenic (αMyHC-FKBP12) mice develop spontaneous atrial fibrillation (AF). The aim of the present study is to explore the mechanisms underlying the occurrence of AF in αMyHC-FKBP12 mice. Spontaneous AF was documented by telemetry in vivo and Langendorff-perfused hearts of αMyHC-FKBP12 and littermate control mice in vitro. Atrial conduction velocity was evaluated by optical mapping. The patch-clamp technique was applied to determine the potentially altered electrophysiology in atrial myocytes. Channel protein expression levels were evaluated by Western blot analyses. Spontaneous AF was recorded in four of seven αMyHC-FKBP12 mice but in none of eight nontransgenic (NTG) controls. Atrial conduction velocity was significantly reduced in αMyHC-FKBP12 hearts compared with NTG hearts. Interestingly, the mean action potential duration at 50% but not 90% was significantly prolonged in αMyHC-FKBP12 atrial myocytes compared with their NTG counterparts. Consistent with decreased conduction velocity, average peak Na+ current ( INa) density was dramatically reduced and the INa inactivation curve was shifted by approximately +7 mV in αMyHC-FKBP12 atrial myocytes, whereas the activation and recovery curves were unaltered. The Nav1.5 expression level was significantly reduced in αMyHC-FKBP12 atria. Furthermore, we found increases in atrial Cav1.2 protein levels and peak L-type Ca2+ current density and increased levels of fibrosis in αMyHC-FKBP12 atria. In summary, cardiomyocyte-restricted overexpression of FKBP12 reduces the atrial Nav1.5 expression level and mean peak INa, which is associated with increased peak L-type Ca2+ current and interstitial fibrosis in atria. The combined electrophysiological and structural changes facilitated the development of local conduction block and altered action potential duration and spontaneous AF. NEW & NOTEWORTHY This study addresses a long-standing riddle regarding the role of FK506-binding protein 12 in cardiac physiology. The work provides further evidence that FK506-binding protein 12 is a critical component for regulating voltage-gated sodium current and in so doing has an important role in arrhythmogenic physiology, such as atrial fibrillation

Leveraging SOLOv2 model to detect heat stress of poultry in complex environments

Heat stress is one of the most important environmental stressors facing poultry production. The presence of heat stress will reduce the antioxidant capacity and immunity of poultry, thereby seriously affecting the health and performance of poultry. The paper proposes an improved FPN-DenseNet-SOLO model for poultry heat stress state detection. The model uses Efficient Channel Attention (ECA) and DropBlock regularization to optimize the DenseNet-169 network to enhance the extraction of poultry heat stress features and suppress the extraction of invalid background features. The model takes the SOLOv2 model as the main frame, and uses the optimized DenseNet-169 as the backbone network to integrate the Feature Pyramid Network to detect and segment instances on the semantic branch and mask branch. In the validation phase, the performance of FPN-DenseNet-SOLO was tested with a test set consisting of 12,740 images of poultry heat stress and normal state, and it was compared with commonly used object detection models (Mask R CNN, Faster RCNN and SOLOv2 model). The results showed that when the DenseNet-169 network lacked the ECA module and the DropBlock regularization module, the original model recognition accuracy was 0.884; when the ECA module was introduced, the model's recognition accuracy improved to 0.919. Not only that, the recall, AP0.5, AP0.75 and mean average precision of the FPN-DenseNet-SOLO model on the test set were all higher than other networks. The recall is 0.954, which is 15, 8.8, and 4.2% higher than the recall of Mask R CNN, Faster R CNN and SOLOv2, respectively. Therefore, the study can achieve accurate segmentation of poultry under normal and heat stress conditions, and provide technical support for the precise breeding of poultry

CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer with Modality-Correlated Cross-Attention for Brain Tumor Segmentation

Brain tumor segmentation (BTS) in magnetic resonance image (MRI) is crucial for brain tumor diagnosis, cancer management and research purposes. With the great success of the ten-year BraTS challenges as well as the advances of CNN and Transformer algorithms, a lot of outstanding BTS models have been proposed to tackle the difficulties of BTS in different technical aspects. However, existing studies hardly consider how to fuse the multi-modality images in a reasonable manner. In this paper, we leverage the clinical knowledge of how radiologists diagnose brain tumors from multiple MRI modalities and propose a clinical knowledge-driven brain tumor segmentation model, called CKD-TransBTS. Instead of directly concatenating all the modalities, we re-organize the input modalities by separating them into two groups according to the imaging principle of MRI. A dual-branch hybrid encoder with the proposed modality-correlated cross-attention block (MCCA) is designed to extract the multi-modality image features. The proposed model inherits the strengths from both Transformer and CNN with the local feature representation ability for precise lesion boundaries and long-range feature extraction for 3D volumetric images. To bridge the gap between Transformer and CNN features, we propose a Trans&CNN Feature Calibration block (TCFC) in the decoder. We compare the proposed model with five CNN-based models and six transformer-based models on the BraTS 2021 challenge dataset. Extensive experiments demonstrate that the proposed model achieves state-of-the-art brain tumor segmentation performance compared with all the competitors

Methylation of SRD5A2 promoter predicts a better outcome for castration-resistant prostate cancer patients undergoing androgen deprivation therapy

PURPOSE: To determine whether SRD5A2 promoter methylation is associated with cancer progression during androgen deprivation therapy (ADT) in CRPC. PATIENTS AND METHODS: In a Local CRPC cohort, 42 prostatic specimens were collected from patients who were diagnosed as CRPC and underwent transurethral resection of the prostate (TURP) at Massachusetts General Hospital (MGH). In a metastatic CRPC (Met CRPC) cohort, 12 metastatic biopsies were collected from CRPC patients who would be treated with abiraterone plus dutasteride (Clinical Trial NCT01393730). As controls, 36 benign prostatic specimens were collected from patients undergoing prostate reduction surgery for symptoms of bladder outlet obstruction secondary to benign prostatic hyperplasia (BPH). The methylation status of cytosine-phosphate-guanine (CpG) site(s) at SRD5A2 promoter regions was tested. RESULTS: Compared with benign prostatic tissue, CRPC samples demonstrated higher SRD5A2 methylation in the whole promoter region (Local CRPC cohort: P \u3c 0.001; Met CRPC cohort: P \u3c 0.05). In Local CRPC cohort, a higher ratio of methylation was correlated with better OS (R2 = 0.33, P = 0.013). Hypermethylation of specific regions (nucleotides -434 to -4 [CpG# -39 to CpG# -2]) was associated with a better OS (11.3+/-5.8 vs 6.4+/-4.4 years, P = 0.001) and PFS (8.4+/-5.4 vs 4.5+/-3.9 years, P = 0.003) with cutoff value of 37.9%. Multivariate analysis showed that SRD5A2 methylation was associated with OS independently (whole promoter region: P = 0.035; specific region: P = 0.02). CONCLUSION: Our study demonstrate that SRD5A2 methylation in promoter regions, specifically at CpG# -39 to -2, is significantly associated with better survival for CRPC patients treated with ADT. Recognition of epigenetic modifications of SRD5A2 may affect the choices and sequence of available therapies for management of CRPC